1. Field of the Invention
This invention relates to a signal multiplexing method and apparatus for multiplexing digital picture or speech signals to generate degree-one multiplexed television-program-based streams, degree-two multiplexing these plural degree-one multiplexed streams to generate a degree-two multiplexed stream to transmit or record the generated degree-two multiplexed stream, a digital signal transmission method and apparatus, a digital signal recording method and apparatus. This invention also relates to a recording medium having the multiplexed data recorded thereon.
2. Description of the Related Art
In ISO 13818-1, there is prescribed a transport stream for multiplexing plural programs, such as television broadcast programs, into a sole multiplexed stream, and transmitting the multiplexed stream. Up to now, there is known a multiplexing system for generating this transport stream.
FIG. 1 shows a structure of a broadcast system transmitting this transport stream using, for example, a digital broadcasting satellite.
To a transmitting device 101, base band video or audio data, associated with plural programs Pa to Pn, are sent from a server or a video camera. These video or audio data are sent to video encoders 102a to 102n and to audio encoders 103a to 103n, associated with the programs Pa to Pn, respectively, so as to be encoded to compressed data streams (elementary streams) conforming to, for example, MPEG (ISO/IEC11172, ISO133818).
The encoded elementary streams are sent to degree-one multiplexers 104a to 104n associated with the programs Pa to Pn, respectively. The degree-one multiplexers 104a to 104n time-divisionally multiplex the respective degree-one multiplexed streams on the transport packet basis to generate a sole degree-two multiplexed stream.
The degree-two multiplexer 105 sends the generated degree-two multiplexed stream via a transmission medium 110 to a reception device 111.
The degree-one multiplexed streams and the degree-two multiplexed stream, obtained on multiplexing by the degree-one multiplexers 104a to 104n and the degree-two multiplexer 105, respectively, conform to the transport stream prescribed in ISO13818-1.
The transmission device 101 thus encodes the base-band video or audio data, associated with the programs Pa to Pn, to generate a sole degree-two multiplexed stream, which is sent to the reception device 111 via transmission medium 110.
The degree-two multiplexed stream is transmitted via the transmission medium 110 to the reception device 111. Specifically, the degree-two multiplexed stream is sent to a separator 112. The separator 112 separates only the elementary stream associated with the program specified by the viewer from the degree-two multiplexed stream to send the separated elementary stream to a decoder. That is, the separator 112 sends the video elementary stream of the specified program to a video decoder 113, while sending the audio elementary stream of the specified program to an audio decoder 114.
The video decoder 113 and the audio decoder 114 expands or decodes the compressed or encoded data to generate baseband video and audio data which are sent to an external equipment, not shown.
The reception device 111 thus receives the supplied degree-two multiplexed stream to select the predetermined program from the plural programs contained in the degree-two multiplexed stream by way of decoding.
FIG. 2 shows data structures of the degree-one multiplexed streams and the degree-two multiplexed stream, as the transport stream prescribed by the ISO13818-1.
The video elementary streams, encoded by the video encoders 102a to 102n, and audio elementary streams, encoded by the audio encoders 103a to 103n, are split into packets termed PES packets. The associated degree-one multiplexers 104a to 104n split the elementary streams into fixed-length transport packets, each being of 188 bytes, and time-divisionally degree-one multiplex the elementary streams on the transport packet basis to generate degree-one multiplexed streams. The degree-two multiplexer 105 time-divisionally degree-two multiplexes the respective degree-one multiplexed streams on the transport packet basis to generate the degree-two multiplexed stream.
FIG. 3 shows a decoder model of the reception device 111 in case the transport stream prescribed by the above-mentioned ISO13818-1 is supplied thereto.
A splitter 112 selects only the transport packets associated with the program and which have been selected by the viewer from the degree-two multiplexed stream to distribute the selected packets to transport buffers 116 to 118. These transport buffers 116 to 118 transiently store the transport packets of the associated data. Specifically, the transport buffer 116 stores the transport packet of video data of the selected program, the transport buffer 117 stores the transport packet of audio data of the selected program and the transport buffer 118 stores the transport packet of program control data of the selected program. Although the transport buffer for title data etc is not shown, transport packets of title data, for example, if contained in the selected program, are stored in the associated transport buffers.
Each of the transport buffers 116 to 118 has a capacity of, for example, 512 bytes, and proceeds to leak the PES packets at a prescribed rate as long as data are stored therein.
The video data leaked from the transport buffer 116 is sent to a multiplexer 119. The audio data and the program control data, leaked from the transport buffers 117, 118, are sent to associated elementary buffers 121, 122.
From the multiplexer 119, only the elementary stream is leaked at a prescribed bitrate so as to be sent to the elementary buffer 120.
At each decoding time point, decoders 123 to 125 extracts elementary streams from the associated elementary buffers 121, 122, every decoding unit (termed an accessing unit) or every picture unit if the data is video data, to carry out decoding. The video data is decoded, using a re-order buffer 126, for chronologically displaying pictures. The base-band video and audio data and control data, thus decoded, are sent to an external equipment or to a system controller.
By the above-described model, the reception device 111 decodes the transport stream prescribed by the above-mentioned ISO13818-1.
In the above-described broadcast system, employing digital satellite broadcast, it is possible to transmit transport streams, obtained on multiplexing data compressed every plural programs by the transmitting device 101, and to demultiplex and decode the data by the reception device 111.
Meanwhile, the total size of the multiplexing buffer 119 and the elementary buffer 120 (121, 122) is the sum of the buffer size of the video buffering verifier (VBV) for controlling the amount of the generated codes and a size uniquely determined by the profile and the level prescribed in ISO13818-2. This sum is referred to hereinafter as a decoder buffer. Thus, each encoder of the transmitting device 101 controls the amount of the generated codes so as to prevent underflow or overflow of the decoder buffer, that is so as not to cause failure of the decoder buffer. The degree-one multiplexers 104a to 104n similarly performs scheduling and multiplexing such as to evade underflow or overflow of the decoder buffer.
Meanwhile, in a multiplexing system used in the above-described broadcast system, that is in a multiplexing system in which degree-one multiplexed streams are further multiplexed to produce a sole degree-two multiplexed stream, there is produced, for the following reason, a time deviation between an output timing of the degree-one multiplexed streams, scheduled by the degree-one multiplexers, and an output timing of the degree-one multiplexed streams multiplexed into the degree-two multiplexed stream.
Specifically, there is produced, in this degree-two multiplexing system, a time deviation between respective output timings of the degree-one multiplexed streams multiplexed into the degree-two multiplexed stream due to the facts that degree-two multiplexing is possible only in transport-packet-based time slots in the bitrate of the degree-two multiplexed stream, the bitrate of the degree-two multiplexed stream is higher than that of the degree-one multiplexed streams, and that, since it is only the transport packet of a sole degree-one multiplexed stream that can be degree-two multiplexed into a given time slot, the transport packet of the degree-one multiplexed stream outputted by the degree-one multiplexer substantially simultaneously is kept in a waiting state. The time deviation between the output timings of the degree-one multiplexed streams scheduled by the respective degree-one multiplexers and the output timings of the degree-one multiplexed streams multiplexed into the degree-two multiplexed stream is termed a degree-two multiplexing jitter.
Thus, even if, in the multiplexing system, the respective degree-one multiplexers are scheduled for multiplexing so as not to cause failure of the decoder buffer provided upstream of the decoder, the decoder buffer is susceptible to failure under the effect of deviation caused by the degree-two multiplexing jitter in the arrival timing of the data stream at the decoder buffer or under the effect of deviation in the time reference encoded in the degree-two multiplexed stream or in the program check reference (PCR) in the case of the transport stream prescribed in ISO13818-1.
These two effects in the multiplexing system are scrutinized in more detail.
Referring first to FIGS. 4A to 4D, the deviation in the arrival timing of the data stream at the decoder buffer due to degree-two multiplexing jitter is explained. Meanwhile, FIGS. 4A to 4D illustrate a degree-one multiplexed stream of a predetermined program into which are multiplexed a video elementary stream and an audio elementary stream, a degree-two multiplexed stream into which are multiplexed the above-mentioned predetermined program and other programs, the buffer occupation volume of the decoder buffer envisaged by the degree-one multiplexers and the buffer occupation volume of the decoder buffer corrupted with the degree-two multiplexing jitter, respectively.
The degree-one multiplexed stream is video or audio data multiplexed every transport packet (N, N+1, N+2, N+3, . . . ) constituted by the packet header and the elementary stream, as shown in FIG. 4A.
The degree-two multiplexed stream is composed of degree-one multiplexed streams shown in FIG. 4A and other degree-one multiplexed streams, degree-two multiplexed at a bitrate higher than the bitrate of the degree-one multiplexed streams, as shown in FIG. 4B.
It is noted that the degree-one multiplexed streams are scheduled and multiplexed so as not to produce overflow or underflow in the decoder buffer. Thus, the video data contained in the pre-set transport packet, such as the transport packet N+1, is decoded without causing failure of the decoder buffer, while audio data contained in the pre-set transport packet, such as the transport packet N+3, is also decoded without causing failure of the decoder buffer, as shown in FIG. 4C.
However, since the degree-two multiplexed stream is of a bitrate higher than the degree-one multiplexed streams, the packet data storage end timing of the degree-two multiplexed stream is occasionally earlier than that of the degree-one multiplexed streams, even if the input start timing of the transport packet into the decoder buffer in the degree-two multiplexed stream is the same as that in the degree-one multiplexed streams. The result is that the end timing of packet data storage into the decoder buffer becomes faster than the timing envisaged by the degree-one multiplexers. Thus, the end timing of packet data storage in the decoder buffer becomes faster than the timing envisaged by the degree-one multiplexers. In such transport packet, such as transport packet N+1, the buffer occupation volume exceeds the decoder buffer size to produce overflow, as shown in FIG. 4D.
Also, since the degree-two multiplexed stream is corrupted with the degree-two multiplexing jitter, the transport packet input start timing to the decoder buffer is occasionally later than that in the degree-one multiplexed streams, such that the packet data storage start timing to the decoder buffer is later than the timing envisaged by the degree-one multiplexed streams. In such transport packet, such as transport packet N+3, the buffer occupation volume is smaller than zero, as shown in FIG. 4D, thus producing the underflow.
Next, description is made of the deviation produced in the time reference value (PCR) encoded in the degree-two multiplexed stream due to the degree-two multiplexing jitter.
The PCR is encoded into the packet header of a predetermined transport packet. This PCR denotes the data input time to a decoder. The decoder of the reception device is actuated in synchronism with system clocks of the reception device based on the actual input timing of byte data including the last bit of the PCR base field and on the PCR value. That is, both the value itself of the PCR and the arrival timing of the PCR at the decoder, that is the timing of actual output timing of the multiplexer, are meaningful. Therefore, if degree-two multiplexing jitter is produced in the transport packet, into which is encoded the PCR, such as the transport packet N+2 shown in FIG. 4, there is produced time deviation between the PCR output timing and the value of the PCR′ shown in FIG. 4B, such that synchronization cannot be achieved in the decoder. The ISO13138-1 provides that only the PCR deviation up to ±500 nanosecond is allowed.
Thus, in the conventional multiplexing system, the following processing is generally adopted in order to overcome the inconveniences of deviation in data stream arrival timing at the decoder buffer and of deviation of the time reference PCR encoded in the degree-two multiplexed stream.
That is, in the conventional multiplexing system, an imaginary buffer occupation volume of the decoder buffer is presumed, in generating the degree-one multiplexed streams by the degree-one multiplexers, and a pre-set margin is provided for each of the upper and lower values of the buffer occupation volume of the decoder buffer in order to effect multiplexing. The result is that, in the conventional multiplexing system, such a degree-two multiplexed stream is produced which does not produce failure in the decoder buffer even if the data stream arrival timing at the decoder buffer is earlier or later due to the degree-two multiplexing jitter. Also, in the conventional multiplexing system, a PCR correction unit for correcting the PCR of the degree-two multiplexed stream is provided downstream of the degree-two multiplexer. The result is that if, in the conventional multiplexing system, the PCR is deviated due to the degree-two multiplexing jitter, such a degree-two multiplexed stream can be realized which enables synchronization to be achieved at the decoder of the reception device.
Referring to FIG. 5, a conventional multiplexing system 150 which overcomes these two problems is explained.
The conventional multiplexing system 150 includes plural degree-one multiplexers 151a to 151n and a degree-two multiplexer 152. This degree-two multiplexer 152 is made up of reception memories 153a to 153n, associated with the degree-one multiplexers 151a to 151n, respectively, a switching unit 154 for switching between the degree-one multiplexed streams stored in the reception memories 153a to 153n to multiplex the selected degree-one multiplexed streams, a controller 155 for controlling the switching unit 154 and a time information correction unit 156 for correcting the PCR of the degree-two multiplexed stream generated by the switching unit 154.
The degree-one multiplexers 151a to 151n provide a pre-set margin for each of the upper and lower values of the buffer occupation volume of the data buffer to generate degree-one multiplexed streams of code volumes which do not cause the failure of the decoder buffer. The degree-one multiplexed streams are sent to the associated reception memories 153a to 153n. 
The reception memories 153a to 153n transiently store the degree-one multiplexed streams generated by the degree-one multiplexers 151a to 151n. 
The controller 155 monitors the memory occupation volumes of the reception memories 153a to 153n to select the degree-one multiplexed streams to be degree-two multiplexed in the arrival sequence of the degree-one multiplexed streams or in the sequence of the decreasing memory occupation volumes to control the switching unit 154.
The switching unit 154 switches between the reception memories 153a to 153n to multiplex the selected streams, under control by the controller 155, in order to generate the degree-two multiplexed stream.
The time information correction unit 156 rewrites the value of the PCR of the degree-two multiplexed stream generated by the switching unit 154 to a timing of the actual occurrence of the degree-two multiplexing.
Thus, with the conventional multiplexing system 150, the input data is multiplexed by the degree-one multiplexers 151a to 151n with a margin so as not to cause failure of the decoder buffer, and the PCR value is rewritten by the time information correction unit 156, in order to generate the degree-two multiplexed stream which does not cause failure of the decoder buffer and which enables synchronization to be achieved with the transmission device by the decoder of the reception device.
In the above-described conventional multiplexing system 150, the controller 155 monitors the memory occupation volumes of the reception memories 153a to 153n to select the degree-one multiplexed streams for degree-two multiplexing in the sequence of arrival of the degree-one multiplexed streams or in the decreasing order of the memory occupation volumes. The result is that the degree-one multiplexed streams needs to be sent in real-time from the degree-one multiplexers 151a to 151n to the reception memories 153a to 153n. That is, if the transfer to the reception memories 153a to 153n is delayed, the priority sequence of the transport packet of the delayed degree-one multiplexed stream is lower than that of the remaining degree-one multiplexed streams, thus producing significant degree-two multiplexing jitter.
The result is that, in the conventional multiplexing system 150, the degree-one multiplexed streams need to be transferred from the degree-one multiplexers 151a to 151n to the reception memories 153a to 153n by a dedicated transfer system, such that the burst-like data transfer system, such as peripheral component interconnect (PCI) system, used in computers in general, cannot be used.
Also, if, in the conventional multiplexing system 150, the transport packets of all degree-one multiplexed streams arrive at the reception memories 152a to 152n substantially simultaneously, there is produced the degree-two multiplexing jitter corresponding to (I−1) packets at the maximum, where I is the number of the degree-one multiplexed streams, on multiplexing the degree-one multiplexed streams in the degree-two multiplexed stream. The result is the increased margin of the buffer occupation volume of the decoder buffer provided for multiplexing by the degree-one multiplexers 1151a to 151n, thus the degree of freedom in multiplexing being lost.
Moreover, in the conventional multiplexing system 150, the system configuration is complex because the PCR is corrected by the time information correction unit 156.